None
The library imports relevant to this notebook are aready taken care of by importing PTT.
NOTE: This notebook assumes that the pipeline version to be tested is already installed and its environment is activated.
To be able to run this notebook you need to install nptt.
If all goes well you will be able to import PTT.
# Create a temporary directory to hold notebook output, and change the working directory to that directory.
from tempfile import TemporaryDirectory
import os
import shutil
data_dir = TemporaryDirectory()
os.chdir(data_dir.name)
import os
if 'CRDS_CACHE_TYPE' in os.environ:
if os.environ['CRDS_CACHE_TYPE'] == 'local':
os.environ['CRDS_PATH'] = os.path.join(os.environ['HOME'], 'crds', 'cache')
elif os.path.isdir(os.environ['CRDS_CACHE_TYPE']):
os.environ['CRDS_PATH'] = os.environ['CRDS_CACHE_TYPE']
print('CRDS cache location: {}'.format(os.environ['CRDS_PATH']))
CRDS cache location: /grp/crds/cache
import warnings
import psutil
from astropy.io import fits
# Only print a DeprecationWarning the first time it shows up, not every time.
with warnings.catch_warnings():
warnings.simplefilter("once", category=DeprecationWarning)
import jwst
from jwst.pipeline.calwebb_detector1 import Detector1Pipeline
from jwst.assign_wcs.assign_wcs_step import AssignWcsStep
from jwst.msaflagopen.msaflagopen_step import MSAFlagOpenStep
from jwst.extract_2d.extract_2d_step import Extract2dStep
from jwst.srctype.srctype_step import SourceTypeStep
from jwst.wavecorr.wavecorr_step import WavecorrStep
from jwst.flatfield.flat_field_step import FlatFieldStep
from jwst.pathloss.pathloss_step import PathLossStep
from jwst import datamodels
# The latest version of NPTT is installed in the requirements text file at:
# /jwst_validation_notebooks/environment.yml
# import NPTT
import nirspec_pipe_testing_tool as nptt
# To get data from Artifactory
from ci_watson.artifactory_helpers import get_bigdata
# Make sure that the version used is the right one
pipeline_version = jwst.__version__
nptt_version = nptt.__version__
print("Using jwst pipeline version: ", pipeline_version)
print("Using NPTT version: ", nptt_version)
Using jwst pipeline version: 1.8.2 Using NPTT version: 2.0.1
The NIRSpec team developed an implementation of the pathloss step and compared the output to the pipeline output for this step. We expected machine precision differences (about 10^-7) but due to rounding differences in interpolation methods we relaxed the difference threshold to 10^-6.
For the test to be considered PASSED, every single slit (for FS data), slitlet (for MOS data) or slice (for IFU data) in the input file has to pass. If there is any failure, the whole test will be considered as FAILED.
The code for this test for Fixed Slits (FS) can be obtained from: https://github.com/spacetelescope/nirspec_pipe_testing_tool/blob/master/nirspec_pipe_testing_tool/calwebb_spec2_pytests/auxiliary_code/pathloss_fs_ps.py. There is a corresponding script for Multi Object Spectroscopy (MOS) and for Integral Field Unit (IFU) data. Each mode has a script for point source (ps) and for uniform source (uni).
Step description: https://jwst-pipeline.readthedocs.io/en/latest/jwst/pathloss/description.html
Pipeline code: https://github.com/spacetelescope/jwst/tree/master/jwst/pathloss
If the test PASSED this means that all slits, slitlets, or slices individually passed the test. However, if ony one individual slit (for FS data), slitlet (for MOS data) or slice (for IFU data) test failed, the whole test will be reported as FAILED.
A short description and link to the page: https://outerspace.stsci.edu/display/JWSTCC/Vanilla+Spectral+GWCS+Information
Acronymns used un this notebook:
pipeline: calibration pipeline
spec2: spectroscopic calibration pipeline level 2b
PTT: NIRSpec pipeline testing tool (https://github.com/spacetelescope/nirspec_pipe_testing_tool)
The pipeline can be run from the command line in two variants: full or per step.
Tu run the spec2 pipeline in full use the command:
$ strun jwst.pipeline.Spec2Pipeline jwtest_rate.fits
Tu only run the pathloss step, use the command:
$ strun jwst.pathloss.PathLossStep jwtest_previous_step_output.fits
These options are also callable from a script with the testing environment active. The Python call for running the pipeline in full or by step are:
$\gt$ from jwst.pipeline.calwebb_spec2 import Spec2Pipeline
$\gt$ Spec2Pipeline.call(jwtest_rate.fits)
or
$\gt$ from jwst.pathloss import PathLossStep
$\gt$ PathLossStep.call(jwtest_previous_step_output.fits)
PTT can run the spec2 pipeline either in full or per step, as well as the imaging pipeline in full. In this notebook we will use PTT to run the pipeline and the validation tests. To run PTT, follow the directions in the corresponding repo page.
All testing data is from the CV3 campaign. We chose these files because this is our most complete data set, i.e. all modes and filter-grating combinations.
Data used was for testing was only FS and MOS, since extract_2d is skipped for IFU. Data sets are:
testing_data = {
'ifu_g395h_f290lp':{
'uncal_file_nrs1': 'ifu_g395h_f290lp_nrs1_uncal.fits',
'uncal_file_nrs2': 'ifu_g395h_f290lp_nrs2_uncal.fits',
'msa_shutter_config': None }
}
# define function to pull data from Artifactory
def get_artifactory_file(data_set_dict, detector):
"""This function creates a list with all the files needed per detector to run the test.
Args:
data_set_dict: dictionary, contains inputs for a specific mode and configuration
detector: string, either nrs1 or nrs2
Returns:
data: list, contains all files needed to run test
"""
files2obtain = ['uncal_file_nrs1', 'msa_shutter_config']
data = []
for file in files2obtain:
data_file = None
try:
if '_nrs' in file and '2' in detector:
file = file.replace('_nrs1', '_nrs2')
data_file = get_bigdata('jwst_validation_notebooks',
'validation_data',
'nirspec_data',
data_set_dict[file])
except TypeError:
data.append(None)
continue
data.append(data_file)
return data
# Set common NPTT switches for this test
# accepted threshold difference with respect to benchmark files
threshold_diff = 9.999e-05
# other NPTT variables
writefile = False
show_figs = True
save_figs = False
# Get the data
detectors = ['nrs1', 'nrs2']
results_dict = {}
for mode_config, data_set_dict in testing_data.items():
for det in detectors:
print('Testing files for detector: ', det)
data = get_artifactory_file(data_set_dict, det)
uncal_file, msa_shutter_config = data
print('Working with uncal_file: ', uncal_file)
uncal_basename = os.path.basename(uncal_file)
# Run the stage 1 pipeline
rate_object = Detector1Pipeline.call(uncal_file)
# Make sure the MSA shutter configuration file is set up correctly
rate_object.meta.instrument.msa_metadata_file = msa_shutter_config
if msa_shutter_config is not None:
rate_object.meta.instrument.msa_metadata_file = msa_shutter_config
print('MSA shutter configuration file: ', rate_object.meta.instrument.msa_metadata_file)
# Run the stage 2 pipeline steps
try:
pipe_object = AssignWcsStep.call(rate_object)
skip_file = False
except:
print('An error occurred that made the pipeline crash, probably: No open slits fall on detector ', det)
print("Skipping test for this file. \n")
skip_file = True
if not skip_file:
if 'ifu' in uncal_basename.lower() or 'mos' in uncal_basename.lower():
pipe_object = MSAFlagOpenStep.call(pipe_object)
if 'ifu' not in uncal_basename.lower():
pipe_object = Extract2dStep.call(pipe_object)
pipe_object = SourceTypeStep.call(pipe_object)
pipe_object = WavecorrStep.call(pipe_object)
flat_field_object = FlatFieldStep.call(pipe_object)
pathloss_object = PathLossStep.call(flat_field_object)
# get the corresponding reference file from the pathloss datamodel
reffile_path = pathloss_object.meta.ref_file.pathloss.name.replace("crds://",
"https://jwst-crds.stsci.edu/unchecked_get/references/jwst/")
# Run the validation test
%matplotlib inline
if 'fs' in uncal_file.lower():
print('Running test for FS...')
source_type = pathloss_object.meta.target.source_type
if source_type == 'POINT' or source_type is None:
median_diff, result_msg, log_msgs = nptt.calwebb_spec2_pytests.auxiliary_code.pathloss_fs_ps.pathtest(
flat_field_object,
reffile_path,
pathloss_object,
writefile=writefile,
show_figs=show_figs,
save_figs=save_figs,
threshold_diff=threshold_diff)
else:
median_diff, result_msg, log_msgs = nptt.calwebb_spec2_pytests.auxiliary_code.pathloss_fs_uni.pathtest(
flat_field_object,
reffile_path,
pathloss_object,
writefile=writefile,
show_figs=show_figs,
save_figs=save_figs,
threshold_diff=threshold_diff)
if 'mos' in uncal_file.lower():
print('Running test for MOS...')
median_diff, result_msg, log_msgs = nptt.calwebb_spec2_pytests.auxiliary_code.pathloss_mos.pathtest(
flat_field_object,
reffile_path,
pathloss_object,
writefile=writefile,
show_figs=show_figs,
save_figs=save_figs,
threshold_diff=threshold_diff)
if 'ifu' in uncal_file.lower():
print('Running test for IFU...')
if pathloss_object.meta.target.source_type == 'POINT':
median_diff, result_msg, log_msgs = nptt.calwebb_spec2_pytests.auxiliary_code.pathloss_ifu_ps.pathtest(
flat_field_object,
reffile_path,
pathloss_object,
writefile=writefile,
show_figs=show_figs,
save_figs=save_figs,
threshold_diff=threshold_diff)
else:
median_diff, result_msg, log_msgs = nptt.calwebb_spec2_pytests.auxiliary_code.pathloss_ifu_uni.pathtest(
flat_field_object,
reffile_path,
pathloss_object,
writefile=writefile,
show_figs=show_figs,
save_figs=save_figs,
threshold_diff=threshold_diff)
else:
result_msg = 'skipped'
# Did the test passed
print("Did pathloss validation test passed? ", result_msg, "\n\n")
rd = {uncal_basename: result_msg}
results_dict.update(rd)
# close all open files
psutil.Process().open_files()
closing_files = []
for fd in psutil.Process().open_files():
if data_dir.name in fd.path:
closing_files.append(fd)
for fd in closing_files:
try:
print('Closing file: ', fd)
open(fd.fd).close()
except:
print('File already closed: ', fd)
Testing files for detector: nrs1 Working with uncal_file: /internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf/ifu_g395h_f290lp_nrs1_uncal.fits
2022-12-04 01:59:50,989 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /grp/crds/cache/references/jwst/jwst_nirspec_pars-detector1pipeline_0004.asdf
2022-12-04 01:59:51,020 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2022-12-04 01:59:51,022 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2022-12-04 01:59:51,023 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2022-12-04 01:59:51,024 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2022-12-04 01:59:51,025 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2022-12-04 01:59:51,026 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2022-12-04 01:59:51,027 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2022-12-04 01:59:51,028 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2022-12-04 01:59:51,029 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2022-12-04 01:59:51,030 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2022-12-04 01:59:51,032 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2022-12-04 01:59:51,033 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2022-12-04 01:59:51,034 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2022-12-04 01:59:51,035 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2022-12-04 01:59:51,037 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2022-12-04 01:59:51,038 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2022-12-04 01:59:51,039 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2022-12-04 01:59:51,149 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf/ifu_g395h_f290lp_nrs1_uncal.fits',).
2022-12-04 01:59:51,159 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_calibrated_ramp': False, 'steps': {'group_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dq_init': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'saturation': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'n_pix_grow_sat': 1}, 'ipc': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'superbias': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'refpix': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}, 'rscd': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'type': 'baseline'}, 'firstframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'lastframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'linearity': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dark_current': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'dark_output': None}, 'reset': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'persistence': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'input_trapsfilled': '', 'flag_pers_cutoff': 40.0, 'save_persistence': False, 'save_trapsfilled': True}, 'jump': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0, 'after_jump_flag_dn1': 0.0, 'after_jump_flag_time1': 0.0, 'after_jump_flag_dn2': 0.0, 'after_jump_flag_time2': 0.0, 'min_sat_area': 1.0, 'min_jump_area': 5.0, 'expand_factor': 2.0, 'use_ellipses': False, 'sat_required_snowball': True, 'expand_large_events': False}, 'ramp_fit': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'int_name': '', 'save_opt': False, 'opt_name': '', 'suppress_one_group': True, 'maximum_cores': 'none'}, 'gain_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}}}
2022-12-04 01:59:51,561 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'ifu_g395h_f290lp_nrs1_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'persat', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias', 'trapdensity', 'trappars']
2022-12-04 01:59:51,572 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_dark_0088.fits'.
2022-12-04 01:59:51,574 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_gain_0023.fits'.
2022-12-04 01:59:51,575 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_linearity_0022.fits'.
2022-12-04 01:59:51,576 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_mask_0010.fits'.
2022-12-04 01:59:51,579 - stpipe.Detector1Pipeline - INFO - Prefetch for PERSAT reference file is 'N/A'.
2022-12-04 01:59:51,579 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0018.fits'.
2022-12-04 01:59:51,581 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_refpix_0019.fits'.
2022-12-04 01:59:51,582 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2022-12-04 01:59:51,583 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2022-12-04 01:59:51,583 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_saturation_0031.fits'.
2022-12-04 01:59:51,585 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_superbias_0117.fits'.
2022-12-04 01:59:51,588 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPDENSITY reference file is 'N/A'.
2022-12-04 01:59:51,589 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPPARS reference file is 'N/A'.
2022-12-04 01:59:51,589 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2022-12-04 01:59:52,424 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 01:59:52,425 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 01:59:52,920 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2022-12-04 01:59:52,921 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2022-12-04 01:59:52,924 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2022-12-04 01:59:53,054 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 01:59:53,055 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 01:59:53,077 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /grp/crds/cache/references/jwst/jwst_nirspec_mask_0010.fits
2022-12-04 01:59:54,668 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2022-12-04 01:59:54,782 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 01:59:54,784 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf', 'n_pix_grow_sat': 1}
2022-12-04 01:59:54,812 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /grp/crds/cache/references/jwst/jwst_nirspec_saturation_0031.fits
2022-12-04 02:00:00,432 - stpipe.Detector1Pipeline.saturation - INFO - Detected 176585 saturated pixels
2022-12-04 02:00:00,576 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2022-12-04 02:00:00,601 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2022-12-04 02:00:00,717 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:00:00,718 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:00:00,719 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2022-12-04 02:00:00,722 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc done
2022-12-04 02:00:00,833 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:00:00,834 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:00:00,861 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /grp/crds/cache/references/jwst/jwst_nirspec_superbias_0117.fits
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stdatamodels/validate.py:38: ValidationWarning: While validating meta.exposure.readpatt the following error occurred:
'ALLIRS2' is not one of ['ACQ1', 'ACQ2', 'BRIGHT1', 'BRIGHT2', 'DEEP2', 'DEEP8', 'FAST', 'FASTGRPAVG', 'FASTGRPAVG8', 'FASTGRPAVG16', 'FASTGRPAVG32', 'FASTGRPAVG64', 'FASTR1', 'FASTR100', 'FGS', 'FGS60', 'FGS8370', 'FGS840', 'FGSRAPID', 'FINEGUIDE', 'ID', 'MEDIUM2', 'MEDIUM8', 'NIS', 'NISRAPID', 'NRS', 'NRSIRS2', 'NRSN16R4', 'NRSN32R8', 'NRSN8R2', 'NRSRAPID', 'NRSIRS2RAPID', 'NRSRAPIDD1', 'NRSRAPIDD2', 'NRSRAPIDD6', 'NRSSLOW', 'RAPID', 'SHALLOW2', 'SHALLOW4', 'SLOW', 'SLOWR1', 'TRACK', 'ANY', 'N/A']
Failed validating 'enum' in schema:
OrderedDict([('title', 'Readout pattern'),
('type', 'string'),
('enum',
['ACQ1',
'ACQ2',
'BRIGHT1',
'BRIGHT2',
'DEEP2',
'DEEP8',
'FAST',
'FASTGRPAVG',
'FASTGRPAVG8',
'FASTGRPAVG16',
'FASTGRPAVG32',
'FASTGRPAVG64',
'FASTR1',
'FASTR100',
'FGS',
'FGS60',
'FGS8370',
'FGS840',
'FGSRAPID',
'FINEGUIDE',
'ID',
'MEDIUM2',
'MEDIUM8',
'NIS',
'NISRAPID',
'NRS',
'NRSIRS2',
'NRSN16R4',
'NRSN32R8',
'NRSN8R2',
'NRSRAPID',
'NRSIRS2RAPID',
'NRSRAPIDD1',
'NRSRAPIDD2',
'NRSRAPIDD6',
'NRSSLOW',
'RAPID',
'SHALLOW2',
'SHALLOW4',
'SLOW',
'SLOWR1',
'TRACK',
'ANY',
'N/A']),
('fits_ke ...
warnings.warn(errmsg, ValidationWarning)
2022-12-04 02:00:04,760 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2022-12-04 02:00:04,881 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:00:04,882 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}
2022-12-04 02:00:04,911 - stpipe.Detector1Pipeline.refpix - INFO - Using refpix reference file: /grp/crds/cache/references/jwst/jwst_nirspec_refpix_0019.fits
2022-12-04 02:00:08,174 - stpipe.Detector1Pipeline.refpix - INFO - Working on integration 1
2022-12-04 02:01:19,393 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2022-12-04 02:01:19,635 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:01:19,636 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:01:19,669 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /grp/crds/cache/references/jwst/jwst_nirspec_linearity_0022.fits
2022-12-04 02:01:21,238 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword BAD_LIN_CORR does not correspond to an existing DQ mnemonic, so will be ignored
2022-12-04 02:01:23,064 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2022-12-04 02:01:23,198 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:01:23,200 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf', 'dark_output': None}
2022-12-04 02:01:23,506 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /grp/crds/cache/references/jwst/jwst_nirspec_dark_0088.fits
2022-12-04 02:02:02,739 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=20, nframes=1, groupgap=0
2022-12-04 02:02:02,740 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=65, nframes=1, groupgap=0
2022-12-04 02:02:04,125 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2022-12-04 02:02:04,263 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:02:04,265 - stpipe.Detector1Pipeline.jump - INFO - Step jump parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0, 'after_jump_flag_dn1': 0.0, 'after_jump_flag_time1': 0.0, 'after_jump_flag_dn2': 0.0, 'after_jump_flag_time2': 0.0, 'min_sat_area': 1.0, 'min_jump_area': 5.0, 'expand_factor': 2.0, 'use_ellipses': False, 'sat_required_snowball': True, 'expand_large_events': False}
2022-12-04 02:02:04,412 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2022-12-04 02:02:04,565 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0023.fits
2022-12-04 02:02:04,971 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0018.fits
2022-12-04 02:02:06,344 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2022-12-04 02:02:06,508 - stpipe.Detector1Pipeline.jump - INFO - Working on integration 1:
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stcal/jump/twopoint_difference.py:154: RuntimeWarning: All-NaN slice encountered
max_ratio = np.nanmax(ratio, axis=0)
2022-12-04 02:02:14,378 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier, two-point found 65989 pixels with at least one CR from five or more groups.
2022-12-04 02:02:34,799 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 28.4545 sec
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stcal/jump/jump.py:302: RuntimeWarning: invalid value encountered in divide
data /= gain_2d
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stcal/jump/jump.py:303: RuntimeWarning: invalid value encountered in divide
err /= gain_2d
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stcal/jump/jump.py:304: RuntimeWarning: invalid value encountered in divide
readnoise_2d /= gain_2d
2022-12-04 02:02:35,003 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 30.590791
2022-12-04 02:02:35,009 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2022-12-04 02:02:35,141 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:02:35,143 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf', 'int_name': '', 'save_opt': False, 'opt_name': '', 'suppress_one_group': True, 'maximum_cores': 'none'}
2022-12-04 02:02:35,538 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0018.fits
2022-12-04 02:02:35,539 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0023.fits
2022-12-04 02:02:35,633 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = ols
2022-12-04 02:02:35,633 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stcal/ramp_fitting/ols_fit.py:1089: RuntimeWarning: invalid value encountered in multiply
var_p4[num_int, :, :, :] *= (segs_4[num_int, :, :, :] > 0)
2022-12-04 02:05:25,657 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of groups per integration: 20
2022-12-04 02:05:25,658 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of integrations: 1
2022-12-04 02:05:25,862 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2022-12-04 02:05:26,018 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:05:26,019 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scale', 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:05:26,077 - stpipe.Detector1Pipeline.gain_scale - INFO - Rescaling by 1.0
2022-12-04 02:05:26,094 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2022-12-04 02:05:26,208 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:05:26,209 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scaleints', 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:05:26,269 - stpipe.Detector1Pipeline.gain_scale - INFO - Rescaling by 1.0
2022-12-04 02:05:26,286 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2022-12-04 02:05:26,286 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2022-12-04 02:05:26,287 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1019.pmap
2022-12-04 02:05:26,287 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2022-12-04 02:05:26,298 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2022-12-04 02:05:26,410 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:05:26,412 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.1, 'sip_degree': None, 'sip_max_inv_pix_error': 0.1, 'sip_inv_degree': None, 'sip_npoints': 12, 'slit_y_low': -0.55, 'slit_y_high': 0.55}
2022-12-04 02:05:26,630 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1448970586061478 deg
2022-12-04 02:05:26,630 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3232757747173309 deg
2022-12-04 02:05:26,631 - stpipe.AssignWcsStep - INFO - theta_y correction: -3.766823889842878e-13 deg
2022-12-04 02:05:26,632 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2022-12-04 02:05:30,031 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_ifu pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0005.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0035.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0022.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': '/grp/crds/cache/references/jwst/jwst_nirspec_ifupost_0004.asdf', 'ifufore': '/grp/crds/cache/references/jwst/jwst_nirspec_ifufore_0003.asdf', 'ifuslicer': '/grp/crds/cache/references/jwst/jwst_nirspec_ifuslicer_0003.asdf'}
2022-12-04 02:05:37,892 - stpipe.AssignWcsStep - INFO - Update S_REGION to POLYGON ICRS 156.176999392 -45.687618156 156.178775679 -45.687618156 156.178775679 -45.686331563 156.176999392 -45.686331563
2022-12-04 02:05:37,895 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2022-12-04 02:05:37,908 - stpipe.AssignWcsStep - INFO - Results used CRDS context: jwst_1019.pmap
2022-12-04 02:05:37,909 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
2022-12-04 02:05:37,922 - stpipe.MSAFlagOpenStep - INFO - MSAFlagOpenStep instance created.
2022-12-04 02:05:38,226 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:05:38,228 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2022-12-04 02:05:38,252 - stpipe.MSAFlagOpenStep - INFO - Using reference file /grp/crds/cache/references/jwst/jwst_nirspec_msaoper_0001.json
2022-12-04 02:05:38,254 - stpipe.JwstStep - INFO - JwstStep instance created.
2022-12-04 02:05:38,468 - stpipe.MSAFlagOpenStep - INFO - gwa_ytilt is 0.1448970586061478 deg
2022-12-04 02:05:38,469 - stpipe.MSAFlagOpenStep - INFO - gwa_xtilt is 0.3232757747173309 deg
2022-12-04 02:05:38,470 - stpipe.MSAFlagOpenStep - INFO - theta_y correction: -3.766823889842878e-13 deg
2022-12-04 02:05:38,471 - stpipe.MSAFlagOpenStep - INFO - theta_x correction: 0.0 deg
2022-12-04 02:05:38,492 - stpipe.MSAFlagOpenStep - INFO - SPORDER= -1, wrange=[2.87e-06, 5.27e-06]
2022-12-04 02:05:38,688 - stpipe.MSAFlagOpenStep - INFO - There are 5 open slits in quadrant 1
2022-12-04 02:05:38,742 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 2
2022-12-04 02:05:38,775 - stpipe.MSAFlagOpenStep - INFO - There are 9 open slits in quadrant 3
2022-12-04 02:05:38,864 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 4
2022-12-04 02:05:38,896 - stpipe.MSAFlagOpenStep - INFO - There are 0 open slits in quadrant 5
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/astropy/modeling/bounding_box.py:83: RuntimeWarning: Invalid interval: upper bound 2047.5 is strictly less than lower bound 2525.1239286219193.
warnings.warn(f"Invalid interval: upper bound {upper} "
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/astropy/modeling/bounding_box.py:83: RuntimeWarning: Invalid interval: upper bound 2047.5 is strictly less than lower bound 2239.8443211081017.
warnings.warn(f"Invalid interval: upper bound {upper} "
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/astropy/modeling/bounding_box.py:83: RuntimeWarning: Invalid interval: upper bound 2047.5 is strictly less than lower bound 2382.5735098584228.
warnings.warn(f"Invalid interval: upper bound {upper} "
2022-12-04 02:05:53,513 - stpipe.MSAFlagOpenStep - INFO - Results used CRDS context: jwst_1019.pmap
2022-12-04 02:05:53,514 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep done
2022-12-04 02:05:53,529 - stpipe.SourceTypeStep - INFO - SourceTypeStep instance created.
2022-12-04 02:05:53,993 - stpipe.SourceTypeStep - INFO - Step SourceTypeStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:05:53,995 - stpipe.SourceTypeStep - INFO - Step SourceTypeStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'source_type': None}
2022-12-04 02:05:54,005 - stpipe.SourceTypeStep - INFO - Input EXP_TYPE is NRS_IFU
2022-12-04 02:05:54,006 - stpipe.SourceTypeStep - INFO - Input SRCTYAPT = UNKNOWN
2022-12-04 02:05:54,006 - stpipe.SourceTypeStep - INFO - Input source type is unknown; setting default SRCTYPE = EXTENDED
2022-12-04 02:05:54,010 - stpipe.SourceTypeStep - INFO - Step SourceTypeStep done
2022-12-04 02:05:54,021 - stpipe.WavecorrStep - INFO - WavecorrStep instance created.
2022-12-04 02:05:54,220 - stpipe.WavecorrStep - INFO - Step WavecorrStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:05:54,221 - stpipe.WavecorrStep - INFO - Step WavecorrStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2022-12-04 02:05:54,230 - stpipe.WavecorrStep - INFO - Skipping wavecorr correction for EXP_TYPE NRS_IFU
2022-12-04 02:05:54,232 - stpipe.WavecorrStep - INFO - Step WavecorrStep done
2022-12-04 02:05:54,243 - stpipe.FlatFieldStep - INFO - FlatFieldStep instance created.
2022-12-04 02:05:54,443 - stpipe.FlatFieldStep - INFO - Step FlatFieldStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:05:54,445 - stpipe.FlatFieldStep - INFO - Step FlatFieldStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_interpolated_flat': False, 'user_supplied_flat': None, 'inverse': False}
2022-12-04 02:13:35,409 - stpipe.FlatFieldStep - INFO - Results used CRDS context: jwst_1019.pmap
2022-12-04 02:13:35,410 - stpipe.FlatFieldStep - INFO - Step FlatFieldStep done
2022-12-04 02:13:35,424 - stpipe.PathLossStep - INFO - PathLossStep instance created.
2022-12-04 02:13:36,266 - stpipe.PathLossStep - INFO - Step PathLossStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs1_uncal.fits>,).
2022-12-04 02:13:36,268 - stpipe.PathLossStep - INFO - Step PathLossStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'inverse': False, 'source_type': None}
2022-12-04 02:13:36,296 - stpipe.PathLossStep - INFO - Using PATHLOSS reference file /grp/crds/cache/references/jwst/jwst_nirspec_pathloss_0003.fits
2022-12-04 02:13:36,341 - stpipe.PathLossStep - INFO - Input exposure type is NRS_IFU
2022-12-04 02:14:48,628 - stpipe.PathLossStep - INFO - Results used CRDS context: jwst_1019.pmap
2022-12-04 02:14:48,629 - stpipe.PathLossStep - INFO - Step PathLossStep done
Running test for IFU... Checking if files exist and obtaining datamodels. This takes a few minutes... from datamodel --> Detector: NRS1 Grating: G395H Filter: F290LP Lamp: REF EXP_TYPE: NRS_IFU Using reference file: https://jwst-crds.stsci.edu/unchecked_get/references/jwst/jwst_nirspec_pathloss_0003.fits got input slices PLCOR_REF.shape (21, 21, 21) Looping through the slices... working with slice 0 SHAPES (50, 1775) (49, 1774) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 781, right 830, top 2048, bottom 274 NEW SHAPE OF SLICE: (49, 1774) and corr_vals.shape: (49, 1774)
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/nirspec_pipe_testing_tool/calwebb_spec2_pytests/auxiliary_code/pathloss_ifu_uni.py:168: RuntimeWarning: invalid value encountered in divide pathloss_divided = comp_sci/previous_sci
working with slice 1 SHAPES (52, 1737) (51, 1737) ALTERED SHAPE OF SLICE: V2 final left 1197, right 1248, top 2048, bottom 311 NEW SHAPE OF SLICE: (51, 1737) and corr_vals.shape: (51, 1737)
working with slice 2 SHAPES (51, 1779) (50, 1778) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 731, right 781, top 2048, bottom 270 NEW SHAPE OF SLICE: (50, 1778) and corr_vals.shape: (50, 1778)
working with slice 3 SHAPES (52, 1733) (51, 1732) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 1246, right 1297, top 2048, bottom 316 NEW SHAPE OF SLICE: (51, 1732) and corr_vals.shape: (51, 1732)
working with slice 4 SHAPES (51, 1783) (50, 1783) ALTERED SHAPE OF SLICE: V2 final left 682, right 732, top 2048, bottom 265 NEW SHAPE OF SLICE: (50, 1783) and corr_vals.shape: (50, 1783)
working with slice 5 SHAPES (51, 1728) (50, 1728) ALTERED SHAPE OF SLICE: V2 final left 1296, right 1346, top 2048, bottom 320 NEW SHAPE OF SLICE: (50, 1728) and corr_vals.shape: (50, 1728)
working with slice 6 SHAPES (50, 1787) (49, 1787) ALTERED SHAPE OF SLICE: V2 final left 633, right 682, top 2048, bottom 261 NEW SHAPE OF SLICE: (49, 1787) and corr_vals.shape: (49, 1787)
working with slice 7 SHAPES (51, 1723) (50, 1723) ALTERED SHAPE OF SLICE: V2 final left 1345, right 1395, top 2048, bottom 325 NEW SHAPE OF SLICE: (50, 1723) and corr_vals.shape: (50, 1723)
working with slice 8 SHAPES (50, 1791) (49, 1791) ALTERED SHAPE OF SLICE: V2 final left 584, right 633, top 2048, bottom 257 NEW SHAPE OF SLICE: (49, 1791) and corr_vals.shape: (49, 1791)
working with slice 9 SHAPES (51, 1719) (50, 1718) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 1394, right 1444, top 2048, bottom 330 NEW SHAPE OF SLICE: (50, 1718) and corr_vals.shape: (50, 1718)
working with slice 10 SHAPES (51, 1795) (50, 1795) ALTERED SHAPE OF SLICE: V2 final left 534, right 584, top 2048, bottom 253 NEW SHAPE OF SLICE: (50, 1795) and corr_vals.shape: (50, 1795)
working with slice 11 SHAPES (52, 1714) (51, 1713) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 1442, right 1493, top 2048, bottom 335 NEW SHAPE OF SLICE: (51, 1713) and corr_vals.shape: (51, 1713)
working with slice 12 SHAPES (51, 1799) (50, 1799) ALTERED SHAPE OF SLICE: V2 final left 485, right 535, top 2048, bottom 249 NEW SHAPE OF SLICE: (50, 1799) and corr_vals.shape: (50, 1799)
working with slice 13 SHAPES (52, 1709) (51, 1709) ALTERED SHAPE OF SLICE: V2 final left 1491, right 1542, top 2048, bottom 339 NEW SHAPE OF SLICE: (51, 1709) and corr_vals.shape: (51, 1709)
working with slice 14 SHAPES (50, 1803) (49, 1803) ALTERED SHAPE OF SLICE: V2 final left 436, right 485, top 2048, bottom 245 NEW SHAPE OF SLICE: (49, 1803) and corr_vals.shape: (49, 1803)
working with slice 15 SHAPES (52, 1704) (51, 1704) ALTERED SHAPE OF SLICE: V2 final left 1540, right 1591, top 2048, bottom 344 NEW SHAPE OF SLICE: (51, 1704) and corr_vals.shape: (51, 1704)
working with slice 16 SHAPES (50, 1807) (49, 1807) ALTERED SHAPE OF SLICE: V2 final left 387, right 436, top 2048, bottom 241 NEW SHAPE OF SLICE: (49, 1807) and corr_vals.shape: (49, 1807)
working with slice 17 SHAPES (52, 1699) (51, 1699) ALTERED SHAPE OF SLICE: V2 final left 1589, right 1640, top 2048, bottom 349 NEW SHAPE OF SLICE: (51, 1699) and corr_vals.shape: (51, 1699)
working with slice 18 SHAPES (51, 1811) (50, 1811) ALTERED SHAPE OF SLICE: V2 final left 337, right 387, top 2048, bottom 237 NEW SHAPE OF SLICE: (50, 1811) and corr_vals.shape: (50, 1811)
working with slice 19 SHAPES (52, 1695) (51, 1694) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 1638, right 1689, top 2048, bottom 354 NEW SHAPE OF SLICE: (51, 1694) and corr_vals.shape: (51, 1694)
working with slice 20 SHAPES (50, 1815) (49, 1814) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 288, right 337, top 2048, bottom 234 NEW SHAPE OF SLICE: (49, 1814) and corr_vals.shape: (49, 1814)
working with slice 21 SHAPES (52, 1690) (51, 1689) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 1687, right 1738, top 2048, bottom 359 NEW SHAPE OF SLICE: (51, 1689) and corr_vals.shape: (51, 1689)
working with slice 22 SHAPES (51, 1819) (50, 1818) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 238, right 288, top 2048, bottom 230 NEW SHAPE OF SLICE: (50, 1818) and corr_vals.shape: (50, 1818)
working with slice 23 SHAPES (52, 1685) (51, 1684) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 1736, right 1787, top 2048, bottom 364 NEW SHAPE OF SLICE: (51, 1684) and corr_vals.shape: (51, 1684)
working with slice 24 SHAPES (51, 1823) (50, 1822) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 189, right 239, top 2048, bottom 226 NEW SHAPE OF SLICE: (50, 1822) and corr_vals.shape: (50, 1822)
working with slice 25 SHAPES (53, 1680) (52, 1679) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 1784, right 1836, top 2048, bottom 369 NEW SHAPE OF SLICE: (52, 1679) and corr_vals.shape: (52, 1679)
working with slice 26 SHAPES (50, 1826) (49, 1826) ALTERED SHAPE OF SLICE: V2 final left 140, right 189, top 2048, bottom 222 NEW SHAPE OF SLICE: (49, 1826) and corr_vals.shape: (49, 1826)
working with slice 27 SHAPES (53, 1675) (52, 1674) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V6 final left 1833, right 1885, top 2048, bottom 374 NEW SHAPE OF SLICE: (52, 1674) and corr_vals.shape: (52, 1674)
working with slice 28 SHAPES (50, 1830) (49, 1830) ALTERED SHAPE OF SLICE: V2 final left 90, right 139, top 2048, bottom 218 NEW SHAPE OF SLICE: (49, 1830) and corr_vals.shape: (49, 1830)
working with slice 29 SHAPES (52, 1669) (51, 1669) ALTERED SHAPE OF SLICE: V2 final left 1882, right 1933, top 2048, bottom 379 NEW SHAPE OF SLICE: (51, 1669) and corr_vals.shape: (51, 1669)
Calculating statistics...
Absolute Difference : mean = 0.000e+00 median = 0.000e+00 stdev = 0.000e+00
Maximum AbsoluteDifference = 0.000e+00
Minimum AbsoluteDifference = 0.000e+00
Percentage of pixels where median of absolute differences is greater than:
-> 1xtheshold = 0%
-> 3xtheshold = 0%
-> 5xtheshold = 0%
*** Result of the test: PASSED
*** Final result of pathloss test is PASSED ***
('* Script ifu_uni.py took ', '1.5187095522880554 minutes to finish.')
Did pathloss validation test passed? All slits PASSED path_loss test.
Testing files for detector: nrs2
Working with uncal_file: /internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf/ifu_g395h_f290lp_nrs2_uncal.fits
2022-12-04 02:16:22,046 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /grp/crds/cache/references/jwst/jwst_nirspec_pars-detector1pipeline_0004.asdf
2022-12-04 02:16:22,083 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2022-12-04 02:16:22,084 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2022-12-04 02:16:22,085 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2022-12-04 02:16:22,086 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2022-12-04 02:16:22,087 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2022-12-04 02:16:22,089 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2022-12-04 02:16:22,090 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2022-12-04 02:16:22,091 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2022-12-04 02:16:22,091 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2022-12-04 02:16:22,093 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2022-12-04 02:16:22,094 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2022-12-04 02:16:22,095 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2022-12-04 02:16:22,096 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2022-12-04 02:16:22,097 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2022-12-04 02:16:22,099 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2022-12-04 02:16:22,100 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2022-12-04 02:16:22,102 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2022-12-04 02:16:23,109 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf/ifu_g395h_f290lp_nrs2_uncal.fits',).
2022-12-04 02:16:23,119 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_calibrated_ramp': False, 'steps': {'group_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dq_init': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'saturation': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'n_pix_grow_sat': 1}, 'ipc': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'superbias': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'refpix': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}, 'rscd': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'type': 'baseline'}, 'firstframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'lastframe': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'linearity': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'dark_current': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'dark_output': None}, 'reset': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}, 'persistence': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'input_trapsfilled': '', 'flag_pers_cutoff': 40.0, 'save_persistence': False, 'save_trapsfilled': True}, 'jump': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0, 'after_jump_flag_dn1': 0.0, 'after_jump_flag_time1': 0.0, 'after_jump_flag_dn2': 0.0, 'after_jump_flag_time2': 0.0, 'min_sat_area': 1.0, 'min_jump_area': 5.0, 'expand_factor': 2.0, 'use_ellipses': False, 'sat_required_snowball': True, 'expand_large_events': False}, 'ramp_fit': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'int_name': '', 'save_opt': False, 'opt_name': '', 'suppress_one_group': True, 'maximum_cores': 'none'}, 'gain_scale': {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}}}
2022-12-04 02:16:23,524 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'ifu_g395h_f290lp_nrs2_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'persat', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias', 'trapdensity', 'trappars']
2022-12-04 02:16:23,533 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_dark_0085.fits'.
2022-12-04 02:16:23,535 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_gain_0020.fits'.
2022-12-04 02:16:23,537 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_linearity_0025.fits'.
2022-12-04 02:16:23,539 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_mask_0011.fits'.
2022-12-04 02:16:23,540 - stpipe.Detector1Pipeline - INFO - Prefetch for PERSAT reference file is 'N/A'.
2022-12-04 02:16:23,541 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0017.fits'.
2022-12-04 02:16:23,542 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_refpix_0018.fits'.
2022-12-04 02:16:23,543 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2022-12-04 02:16:23,544 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2022-12-04 02:16:23,544 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_saturation_0030.fits'.
2022-12-04 02:16:23,546 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/grp/crds/cache/references/jwst/jwst_nirspec_superbias_0118.fits'.
2022-12-04 02:16:23,547 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPDENSITY reference file is 'N/A'.
2022-12-04 02:16:23,548 - stpipe.Detector1Pipeline - INFO - Prefetch for TRAPPARS reference file is 'N/A'.
2022-12-04 02:16:23,548 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2022-12-04 02:16:24,647 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:16:24,649 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:16:25,135 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2022-12-04 02:16:25,135 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2022-12-04 02:16:25,138 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2022-12-04 02:16:25,538 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:16:25,539 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:16:25,564 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /grp/crds/cache/references/jwst/jwst_nirspec_mask_0011.fits
2022-12-04 02:16:26,993 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2022-12-04 02:16:27,377 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:16:27,378 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf', 'n_pix_grow_sat': 1}
2022-12-04 02:16:27,401 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /grp/crds/cache/references/jwst/jwst_nirspec_saturation_0030.fits
2022-12-04 02:16:32,513 - stpipe.Detector1Pipeline.saturation - INFO - Detected 194308 saturated pixels
2022-12-04 02:16:32,657 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2022-12-04 02:16:32,683 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2022-12-04 02:16:33,071 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:16:33,072 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': True, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:16:33,073 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2022-12-04 02:16:33,075 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc done
2022-12-04 02:16:33,456 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:16:33,457 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:16:33,486 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /grp/crds/cache/references/jwst/jwst_nirspec_superbias_0118.fits
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stdatamodels/validate.py:38: ValidationWarning: While validating meta.exposure.readpatt the following error occurred:
'ALLIRS2' is not one of ['ACQ1', 'ACQ2', 'BRIGHT1', 'BRIGHT2', 'DEEP2', 'DEEP8', 'FAST', 'FASTGRPAVG', 'FASTGRPAVG8', 'FASTGRPAVG16', 'FASTGRPAVG32', 'FASTGRPAVG64', 'FASTR1', 'FASTR100', 'FGS', 'FGS60', 'FGS8370', 'FGS840', 'FGSRAPID', 'FINEGUIDE', 'ID', 'MEDIUM2', 'MEDIUM8', 'NIS', 'NISRAPID', 'NRS', 'NRSIRS2', 'NRSN16R4', 'NRSN32R8', 'NRSN8R2', 'NRSRAPID', 'NRSIRS2RAPID', 'NRSRAPIDD1', 'NRSRAPIDD2', 'NRSRAPIDD6', 'NRSSLOW', 'RAPID', 'SHALLOW2', 'SHALLOW4', 'SLOW', 'SLOWR1', 'TRACK', 'ANY', 'N/A']
Failed validating 'enum' in schema:
OrderedDict([('title', 'Readout pattern'),
('type', 'string'),
('enum',
['ACQ1',
'ACQ2',
'BRIGHT1',
'BRIGHT2',
'DEEP2',
'DEEP8',
'FAST',
'FASTGRPAVG',
'FASTGRPAVG8',
'FASTGRPAVG16',
'FASTGRPAVG32',
'FASTGRPAVG64',
'FASTR1',
'FASTR100',
'FGS',
'FGS60',
'FGS8370',
'FGS840',
'FGSRAPID',
'FINEGUIDE',
'ID',
'MEDIUM2',
'MEDIUM8',
'NIS',
'NISRAPID',
'NRS',
'NRSIRS2',
'NRSN16R4',
'NRSN32R8',
'NRSN8R2',
'NRSRAPID',
'NRSIRS2RAPID',
'NRSRAPIDD1',
'NRSRAPIDD2',
'NRSRAPIDD6',
'NRSSLOW',
'RAPID',
'SHALLOW2',
'SHALLOW4',
'SLOW',
'SLOWR1',
'TRACK',
'ANY',
'N/A']),
('fits_ke ...
warnings.warn(errmsg, ValidationWarning)
2022-12-04 02:16:37,798 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2022-12-04 02:16:38,177 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:16:38,179 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf', 'odd_even_columns': True, 'use_side_ref_pixels': True, 'side_smoothing_length': 11, 'side_gain': 1.0, 'odd_even_rows': True}
2022-12-04 02:16:38,201 - stpipe.Detector1Pipeline.refpix - INFO - Using refpix reference file: /grp/crds/cache/references/jwst/jwst_nirspec_refpix_0018.fits
2022-12-04 02:16:40,669 - stpipe.Detector1Pipeline.refpix - INFO - Working on integration 1
2022-12-04 02:17:52,447 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2022-12-04 02:17:52,927 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:17:52,929 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:17:52,957 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /grp/crds/cache/references/jwst/jwst_nirspec_linearity_0025.fits
2022-12-04 02:17:53,928 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword BAD_LIN_CORR does not correspond to an existing DQ mnemonic, so will be ignored
2022-12-04 02:17:55,692 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2022-12-04 02:17:56,079 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:17:56,080 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf', 'dark_output': None}
2022-12-04 02:17:56,377 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /grp/crds/cache/references/jwst/jwst_nirspec_dark_0085.fits
2022-12-04 02:18:33,356 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=20, nframes=1, groupgap=0
2022-12-04 02:18:33,356 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=65, nframes=1, groupgap=0
2022-12-04 02:18:34,869 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2022-12-04 02:18:35,270 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:18:35,272 - stpipe.Detector1Pipeline.jump - INFO - Step jump parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf', 'rejection_threshold': 4.0, 'three_group_rejection_threshold': 6.0, 'four_group_rejection_threshold': 5.0, 'maximum_cores': 'none', 'flag_4_neighbors': True, 'max_jump_to_flag_neighbors': 1000.0, 'min_jump_to_flag_neighbors': 10.0, 'after_jump_flag_dn1': 0.0, 'after_jump_flag_time1': 0.0, 'after_jump_flag_dn2': 0.0, 'after_jump_flag_time2': 0.0, 'min_sat_area': 1.0, 'min_jump_area': 5.0, 'expand_factor': 2.0, 'use_ellipses': False, 'sat_required_snowball': True, 'expand_large_events': False}
2022-12-04 02:18:35,416 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2022-12-04 02:18:35,580 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0020.fits
2022-12-04 02:18:36,268 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0017.fits
2022-12-04 02:18:38,106 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2022-12-04 02:18:38,274 - stpipe.Detector1Pipeline.jump - INFO - Working on integration 1:
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stcal/jump/twopoint_difference.py:154: RuntimeWarning: All-NaN slice encountered
max_ratio = np.nanmax(ratio, axis=0)
2022-12-04 02:18:46,188 - stpipe.Detector1Pipeline.jump - INFO - From highest outlier, two-point found 69597 pixels with at least one CR from five or more groups.
2022-12-04 02:19:11,935 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 33.828 sec
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stcal/jump/jump.py:302: RuntimeWarning: invalid value encountered in divide
data /= gain_2d
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stcal/jump/jump.py:303: RuntimeWarning: invalid value encountered in divide
err /= gain_2d
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stcal/jump/jump.py:304: RuntimeWarning: invalid value encountered in divide
readnoise_2d /= gain_2d
2022-12-04 02:19:12,136 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 36.720247
2022-12-04 02:19:12,141 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2022-12-04 02:19:12,570 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 20, 3200, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:19:12,571 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf', 'int_name': '', 'save_opt': False, 'opt_name': '', 'suppress_one_group': True, 'maximum_cores': 'none'}
2022-12-04 02:19:12,965 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /grp/crds/cache/references/jwst/jwst_nirspec_readnoise_0017.fits
2022-12-04 02:19:12,966 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /grp/crds/cache/references/jwst/jwst_nirspec_gain_0020.fits
2022-12-04 02:19:13,060 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = ols
2022-12-04 02:19:13,061 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/stcal/ramp_fitting/ols_fit.py:1089: RuntimeWarning: invalid value encountered in multiply
var_p4[num_int, :, :, :] *= (segs_4[num_int, :, :, :] > 0)
2022-12-04 02:21:57,633 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of groups per integration: 20
2022-12-04 02:21:57,634 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of integrations: 1
2022-12-04 02:21:57,824 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2022-12-04 02:21:58,268 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:21:58,269 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scale', 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:21:58,326 - stpipe.Detector1Pipeline.gain_scale - INFO - Rescaling by 1.0
2022-12-04 02:21:58,342 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2022-12-04 02:21:58,732 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:21:58,734 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'gain_scaleints', 'search_output_file': True, 'input_dir': '/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/tmp/tmp10_3l9bf'}
2022-12-04 02:21:58,796 - stpipe.Detector1Pipeline.gain_scale - INFO - Rescaling by 1.0
2022-12-04 02:21:58,813 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2022-12-04 02:21:58,813 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2022-12-04 02:21:58,814 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1019.pmap
2022-12-04 02:21:58,814 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2022-12-04 02:21:58,824 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2022-12-04 02:21:59,195 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:21:59,196 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'sip_approx': True, 'sip_max_pix_error': 0.1, 'sip_degree': None, 'sip_max_inv_pix_error': 0.1, 'sip_inv_degree': None, 'sip_npoints': 12, 'slit_y_low': -0.55, 'slit_y_high': 0.55}
2022-12-04 02:21:59,406 - stpipe.AssignWcsStep - INFO - gwa_ytilt is 0.1448970586061478 deg
2022-12-04 02:21:59,407 - stpipe.AssignWcsStep - INFO - gwa_xtilt is 0.3232757747173309 deg
2022-12-04 02:21:59,407 - stpipe.AssignWcsStep - INFO - theta_y correction: -3.766823889842878e-13 deg
2022-12-04 02:21:59,409 - stpipe.AssignWcsStep - INFO - theta_x correction: 0.0 deg
2022-12-04 02:22:02,598 - stpipe.AssignWcsStep - INFO - Created a NIRSPEC nrs_ifu pipeline with references {'distortion': None, 'filteroffset': None, 'specwcs': None, 'regions': None, 'wavelengthrange': '/grp/crds/cache/references/jwst/jwst_nirspec_wavelengthrange_0005.asdf', 'camera': '/grp/crds/cache/references/jwst/jwst_nirspec_camera_0004.asdf', 'collimator': '/grp/crds/cache/references/jwst/jwst_nirspec_collimator_0004.asdf', 'disperser': '/grp/crds/cache/references/jwst/jwst_nirspec_disperser_0035.asdf', 'fore': '/grp/crds/cache/references/jwst/jwst_nirspec_fore_0022.asdf', 'fpa': '/grp/crds/cache/references/jwst/jwst_nirspec_fpa_0005.asdf', 'msa': '/grp/crds/cache/references/jwst/jwst_nirspec_msa_0005.asdf', 'ote': '/grp/crds/cache/references/jwst/jwst_nirspec_ote_0005.asdf', 'ifupost': '/grp/crds/cache/references/jwst/jwst_nirspec_ifupost_0004.asdf', 'ifufore': '/grp/crds/cache/references/jwst/jwst_nirspec_ifufore_0003.asdf', 'ifuslicer': '/grp/crds/cache/references/jwst/jwst_nirspec_ifuslicer_0003.asdf'}
2022-12-04 02:22:09,698 - stpipe.AssignWcsStep - INFO - Update S_REGION to POLYGON ICRS 156.176996289 -45.687617183 156.178772625 -45.687617183 156.178772625 -45.686330364 156.176996289 -45.686330364
2022-12-04 02:22:09,700 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2022-12-04 02:22:09,714 - stpipe.AssignWcsStep - INFO - Results used CRDS context: jwst_1019.pmap
2022-12-04 02:22:09,715 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
2022-12-04 02:22:09,728 - stpipe.MSAFlagOpenStep - INFO - MSAFlagOpenStep instance created.
2022-12-04 02:22:10,428 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:22:10,430 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2022-12-04 02:22:10,451 - stpipe.MSAFlagOpenStep - INFO - Using reference file /grp/crds/cache/references/jwst/jwst_nirspec_msaoper_0001.json
2022-12-04 02:22:10,452 - stpipe.JwstStep - INFO - JwstStep instance created.
2022-12-04 02:22:10,671 - stpipe.MSAFlagOpenStep - INFO - gwa_ytilt is 0.1448970586061478 deg
2022-12-04 02:22:10,672 - stpipe.MSAFlagOpenStep - INFO - gwa_xtilt is 0.3232757747173309 deg
2022-12-04 02:22:10,673 - stpipe.MSAFlagOpenStep - INFO - theta_y correction: -3.766823889842878e-13 deg
2022-12-04 02:22:10,674 - stpipe.MSAFlagOpenStep - INFO - theta_x correction: 0.0 deg
2022-12-04 02:22:10,691 - stpipe.MSAFlagOpenStep - INFO - SPORDER= -1, wrange=[2.87e-06, 5.27e-06]
2022-12-04 02:22:10,867 - stpipe.MSAFlagOpenStep - INFO - There are 5 open slits in quadrant 1
2022-12-04 02:22:10,913 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 2
2022-12-04 02:22:10,939 - stpipe.MSAFlagOpenStep - INFO - There are 9 open slits in quadrant 3
2022-12-04 02:22:11,017 - stpipe.MSAFlagOpenStep - INFO - There are 3 open slits in quadrant 4
2022-12-04 02:22:11,044 - stpipe.MSAFlagOpenStep - INFO - There are 0 open slits in quadrant 5
2022-12-04 02:22:26,725 - stpipe.MSAFlagOpenStep - INFO - Results used CRDS context: jwst_1019.pmap
2022-12-04 02:22:26,726 - stpipe.MSAFlagOpenStep - INFO - Step MSAFlagOpenStep done
2022-12-04 02:22:26,737 - stpipe.SourceTypeStep - INFO - SourceTypeStep instance created.
2022-12-04 02:22:27,420 - stpipe.SourceTypeStep - INFO - Step SourceTypeStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:22:27,423 - stpipe.SourceTypeStep - INFO - Step SourceTypeStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'source_type': None}
2022-12-04 02:22:27,432 - stpipe.SourceTypeStep - INFO - Input EXP_TYPE is NRS_IFU
2022-12-04 02:22:27,432 - stpipe.SourceTypeStep - INFO - Input SRCTYAPT = UNKNOWN
2022-12-04 02:22:27,433 - stpipe.SourceTypeStep - INFO - Input source type is unknown; setting default SRCTYPE = EXTENDED
2022-12-04 02:22:27,436 - stpipe.SourceTypeStep - INFO - Step SourceTypeStep done
2022-12-04 02:22:27,446 - stpipe.WavecorrStep - INFO - WavecorrStep instance created.
2022-12-04 02:22:27,834 - stpipe.WavecorrStep - INFO - Step WavecorrStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:22:27,836 - stpipe.WavecorrStep - INFO - Step WavecorrStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': ''}
2022-12-04 02:22:27,845 - stpipe.WavecorrStep - INFO - Skipping wavecorr correction for EXP_TYPE NRS_IFU
2022-12-04 02:22:27,848 - stpipe.WavecorrStep - INFO - Step WavecorrStep done
2022-12-04 02:22:27,858 - stpipe.FlatFieldStep - INFO - FlatFieldStep instance created.
2022-12-04 02:22:28,239 - stpipe.FlatFieldStep - INFO - Step FlatFieldStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:22:28,240 - stpipe.FlatFieldStep - INFO - Step FlatFieldStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'save_interpolated_flat': False, 'user_supplied_flat': None, 'inverse': False}
2022-12-04 02:30:03,432 - stpipe.FlatFieldStep - INFO - Results used CRDS context: jwst_1019.pmap
2022-12-04 02:30:03,433 - stpipe.FlatFieldStep - INFO - Step FlatFieldStep done
2022-12-04 02:30:03,446 - stpipe.PathLossStep - INFO - PathLossStep instance created.
2022-12-04 02:30:04,820 - stpipe.PathLossStep - INFO - Step PathLossStep running with args (<IFUImageModel(2048, 2048) from ifu_g395h_f290lp_nrs2_uncal.fits>,).
2022-12-04 02:30:04,823 - stpipe.PathLossStep - INFO - Step PathLossStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'inverse': False, 'source_type': None}
2022-12-04 02:30:04,848 - stpipe.PathLossStep - INFO - Using PATHLOSS reference file /grp/crds/cache/references/jwst/jwst_nirspec_pathloss_0003.fits
2022-12-04 02:30:04,887 - stpipe.PathLossStep - INFO - Input exposure type is NRS_IFU
2022-12-04 02:31:16,540 - stpipe.PathLossStep - INFO - Results used CRDS context: jwst_1019.pmap
2022-12-04 02:31:16,541 - stpipe.PathLossStep - INFO - Step PathLossStep done
Running test for IFU... Checking if files exist and obtaining datamodels. This takes a few minutes... from datamodel --> Detector: NRS2 Grating: G395H Filter: F290LP Lamp: REF EXP_TYPE: NRS_IFU Using reference file: https://jwst-crds.stsci.edu/unchecked_get/references/jwst/jwst_nirspec_pathloss_0003.fits got input slices PLCOR_REF.shape (21, 21, 21) Looping through the slices... working with slice 0 SHAPES (49, 1677) (48, 1677) ALTERED SHAPE OF SLICE: V2 final left 781, right 829, top 1677, bottom 0 NEW SHAPE OF SLICE: (48, 1677) and corr_vals.shape: (48, 1677)
/internal/data1/jenkins/workspace/Notebooks/jwst_validation_notebooks_spacetelescope/miniconda3/envs/jwst_validation_notebooks/lib/python3.9/site-packages/nirspec_pipe_testing_tool/calwebb_spec2_pytests/auxiliary_code/pathloss_ifu_uni.py:168: RuntimeWarning: invalid value encountered in divide pathloss_divided = comp_sci/previous_sci
working with slice 1 SHAPES (52, 1721) (51, 1722) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V8 final left 1197, right 1248, top 1721, bottom -1 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1722) ALTERED SHAPE OF SLICE: V7 final left 1197, right 1248, top 1722, bottom -2 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1722) ALTERED SHAPE OF SLICE: V7 final left 1197, right 1248, top 1723, bottom -3 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1722) ALTERED SHAPE OF SLICE: V7 final left 1197, right 1248, top 1724, bottom -4 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1722) ALTERED SHAPE OF SLICE: V7 final left 1197, right 1248, top 1725, bottom -5 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1722) WARNING: may be in infinite loop! ALTERED SHAPE OF SLICE: V7 final left 1197, right 1248, top 1726, bottom -6 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1722) shapes did not match! full_cut2slice: (51, 0), corr_vals (51, 1722) working with slice 2 SHAPES (50, 1672) (49, 1672) ALTERED SHAPE OF SLICE: V2 final left 731, right 780, top 1672, bottom 0 NEW SHAPE OF SLICE: (49, 1672) and corr_vals.shape: (49, 1672)
working with slice 3 SHAPES (51, 1726) (50, 1727) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V8 final left 1247, right 1297, top 1726, bottom -1 NEW SHAPE OF SLICE: (50, 0) and corr_vals.shape: (50, 1727) ALTERED SHAPE OF SLICE: V8 final left 1247, right 1297, top 1726, bottom -2 NEW SHAPE OF SLICE: (50, 0) and corr_vals.shape: (50, 1727) ALTERED SHAPE OF SLICE: V8 final left 1247, right 1297, top 1726, bottom -3 NEW SHAPE OF SLICE: (50, 0) and corr_vals.shape: (50, 1727) ALTERED SHAPE OF SLICE: V8 final left 1247, right 1297, top 1726, bottom -4 NEW SHAPE OF SLICE: (50, 0) and corr_vals.shape: (50, 1727) ALTERED SHAPE OF SLICE: V8 final left 1247, right 1297, top 1726, bottom -5 NEW SHAPE OF SLICE: (50, 0) and corr_vals.shape: (50, 1727) WARNING: may be in infinite loop! ALTERED SHAPE OF SLICE: V8 final left 1247, right 1297, top 1726, bottom -6 NEW SHAPE OF SLICE: (50, 0) and corr_vals.shape: (50, 1727) shapes did not match! full_cut2slice: (50, 0), corr_vals (50, 1727) working with slice 4 SHAPES (49, 1667) (48, 1667) ALTERED SHAPE OF SLICE: V2 final left 682, right 730, top 1667, bottom 0 NEW SHAPE OF SLICE: (48, 1667) and corr_vals.shape: (48, 1667)
working with slice 5 SHAPES (51, 1732) (50, 1732) ALTERED SHAPE OF SLICE: V2 final left 1296, right 1346, top 1732, bottom 0 NEW SHAPE OF SLICE: (50, 1732) and corr_vals.shape: (50, 1732)
working with slice 6 SHAPES (49, 1662) (48, 1663) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V8 final left 633, right 681, top 1662, bottom -1 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1663) ALTERED SHAPE OF SLICE: V8 final left 633, right 681, top 1662, bottom -2 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1663) ALTERED SHAPE OF SLICE: V8 final left 633, right 681, top 1662, bottom -3 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1663) ALTERED SHAPE OF SLICE: V8 final left 633, right 681, top 1662, bottom -4 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1663) ALTERED SHAPE OF SLICE: V8 final left 633, right 681, top 1662, bottom -5 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1663) WARNING: may be in infinite loop! ALTERED SHAPE OF SLICE: V8 final left 633, right 681, top 1662, bottom -6 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1663) shapes did not match! full_cut2slice: (48, 0), corr_vals (48, 1663) working with slice 7 SHAPES (52, 1737) (51, 1738) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V8 final left 1345, right 1396, top 1737, bottom -1 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1738) ALTERED SHAPE OF SLICE: V7 final left 1345, right 1396, top 1738, bottom -2 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1738) ALTERED SHAPE OF SLICE: V7 final left 1345, right 1396, top 1739, bottom -3 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1738) ALTERED SHAPE OF SLICE: V7 final left 1345, right 1396, top 1740, bottom -4 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1738) ALTERED SHAPE OF SLICE: V7 final left 1345, right 1396, top 1741, bottom -5 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1738) WARNING: may be in infinite loop! ALTERED SHAPE OF SLICE: V7 final left 1345, right 1396, top 1742, bottom -6 NEW SHAPE OF SLICE: (51, 0) and corr_vals.shape: (51, 1738) shapes did not match! full_cut2slice: (51, 0), corr_vals (51, 1738) working with slice 8 SHAPES (49, 1657) (48, 1658) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V8 final left 584, right 632, top 1657, bottom -1 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1658) ALTERED SHAPE OF SLICE: V7 final left 584, right 632, top 1658, bottom -2 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1658) ALTERED SHAPE OF SLICE: V7 final left 584, right 632, top 1659, bottom -3 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1658) ALTERED SHAPE OF SLICE: V7 final left 584, right 632, top 1660, bottom -4 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1658) ALTERED SHAPE OF SLICE: V7 final left 584, right 632, top 1661, bottom -5 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1658) WARNING: may be in infinite loop! ALTERED SHAPE OF SLICE: V7 final left 584, right 632, top 1662, bottom -6 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1658) shapes did not match! full_cut2slice: (48, 0), corr_vals (48, 1658) working with slice 9 SHAPES (52, 1743) (51, 1743) ALTERED SHAPE OF SLICE: V2 final left 1394, right 1445, top 1743, bottom 0 NEW SHAPE OF SLICE: (51, 1743) and corr_vals.shape: (51, 1743)
working with slice 10 SHAPES (49, 1653) (48, 1653) ALTERED SHAPE OF SLICE: V2 final left 534, right 582, top 1653, bottom 0 NEW SHAPE OF SLICE: (48, 1653) and corr_vals.shape: (48, 1653)
working with slice 11 SHAPES (52, 1749) (51, 1749) ALTERED SHAPE OF SLICE: V2 final left 1443, right 1494, top 1749, bottom 0 NEW SHAPE OF SLICE: (51, 1749) and corr_vals.shape: (51, 1749)
working with slice 12 SHAPES (49, 1648) (48, 1648) ALTERED SHAPE OF SLICE: V2 final left 485, right 533, top 1648, bottom 0 NEW SHAPE OF SLICE: (48, 1648) and corr_vals.shape: (48, 1648)
working with slice 13 SHAPES (53, 1754) (52, 1755) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V8 final left 1491, right 1543, top 1754, bottom -1 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1755) ALTERED SHAPE OF SLICE: V8 final left 1491, right 1543, top 1754, bottom -2 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1755) ALTERED SHAPE OF SLICE: V8 final left 1491, right 1543, top 1754, bottom -3 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1755) ALTERED SHAPE OF SLICE: V8 final left 1491, right 1543, top 1754, bottom -4 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1755) ALTERED SHAPE OF SLICE: V8 final left 1491, right 1543, top 1754, bottom -5 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1755) WARNING: may be in infinite loop! ALTERED SHAPE OF SLICE: V8 final left 1491, right 1543, top 1754, bottom -6 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1755) shapes did not match! full_cut2slice: (52, 0), corr_vals (52, 1755) working with slice 14 SHAPES (48, 1643) (47, 1644) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V8 final left 436, right 483, top 1643, bottom -1 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1644) ALTERED SHAPE OF SLICE: V7 final left 436, right 483, top 1644, bottom -2 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1644) ALTERED SHAPE OF SLICE: V7 final left 436, right 483, top 1645, bottom -3 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1644) ALTERED SHAPE OF SLICE: V7 final left 436, right 483, top 1646, bottom -4 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1644) ALTERED SHAPE OF SLICE: V7 final left 436, right 483, top 1647, bottom -5 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1644) WARNING: may be in infinite loop! ALTERED SHAPE OF SLICE: V7 final left 436, right 483, top 1648, bottom -6 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1644) shapes did not match! full_cut2slice: (47, 0), corr_vals (47, 1644) working with slice 15 SHAPES (53, 1760) (52, 1761) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V8 final left 1540, right 1592, top 1760, bottom -1 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1761) ALTERED SHAPE OF SLICE: V8 final left 1540, right 1592, top 1760, bottom -2 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1761) ALTERED SHAPE OF SLICE: V8 final left 1540, right 1592, top 1760, bottom -3 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1761) ALTERED SHAPE OF SLICE: V8 final left 1540, right 1592, top 1760, bottom -4 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1761) ALTERED SHAPE OF SLICE: V8 final left 1540, right 1592, top 1760, bottom -5 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1761) WARNING: may be in infinite loop! ALTERED SHAPE OF SLICE: V8 final left 1540, right 1592, top 1760, bottom -6 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1761) shapes did not match! full_cut2slice: (52, 0), corr_vals (52, 1761) working with slice 16 SHAPES (48, 1639) (47, 1639) ALTERED SHAPE OF SLICE: V2 final left 387, right 434, top 1639, bottom 0 NEW SHAPE OF SLICE: (47, 1639) and corr_vals.shape: (47, 1639)
working with slice 17 SHAPES (53, 1766) (52, 1767) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V8 final left 1589, right 1641, top 1766, bottom -1 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1767) ALTERED SHAPE OF SLICE: V8 final left 1589, right 1641, top 1766, bottom -2 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1767) ALTERED SHAPE OF SLICE: V8 final left 1589, right 1641, top 1766, bottom -3 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1767) ALTERED SHAPE OF SLICE: V8 final left 1589, right 1641, top 1766, bottom -4 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1767) ALTERED SHAPE OF SLICE: V8 final left 1589, right 1641, top 1766, bottom -5 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1767) WARNING: may be in infinite loop! ALTERED SHAPE OF SLICE: V8 final left 1589, right 1641, top 1766, bottom -6 NEW SHAPE OF SLICE: (52, 0) and corr_vals.shape: (52, 1767) shapes did not match! full_cut2slice: (52, 0), corr_vals (52, 1767) working with slice 18 SHAPES (49, 1634) (48, 1635) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V8 final left 337, right 385, top 1634, bottom -1 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1635) ALTERED SHAPE OF SLICE: V8 final left 337, right 385, top 1634, bottom -2 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1635) ALTERED SHAPE OF SLICE: V8 final left 337, right 385, top 1634, bottom -3 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1635) ALTERED SHAPE OF SLICE: V8 final left 337, right 385, top 1634, bottom -4 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1635) ALTERED SHAPE OF SLICE: V8 final left 337, right 385, top 1634, bottom -5 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1635) WARNING: may be in infinite loop! ALTERED SHAPE OF SLICE: V8 final left 337, right 385, top 1634, bottom -6 NEW SHAPE OF SLICE: (48, 0) and corr_vals.shape: (48, 1635) shapes did not match! full_cut2slice: (48, 0), corr_vals (48, 1635) working with slice 19 SHAPES (53, 1772) (52, 1772) ALTERED SHAPE OF SLICE: V2 final left 1638, right 1690, top 1772, bottom 0 NEW SHAPE OF SLICE: (52, 1772) and corr_vals.shape: (52, 1772)
working with slice 20 SHAPES (48, 1630) (47, 1630) ALTERED SHAPE OF SLICE: V2 final left 288, right 335, top 1630, bottom 0 NEW SHAPE OF SLICE: (47, 1630) and corr_vals.shape: (47, 1630)
working with slice 21 SHAPES (54, 1778) (53, 1778) ALTERED SHAPE OF SLICE: V2 final left 1687, right 1740, top 1778, bottom 0 NEW SHAPE OF SLICE: (53, 1778) and corr_vals.shape: (53, 1778)
working with slice 22 SHAPES (48, 1625) (47, 1626) ALTERED SHAPE OF SLICE: V2 ALTERED SHAPE OF SLICE: V8 final left 239, right 286, top 1625, bottom -1 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1626) ALTERED SHAPE OF SLICE: V7 final left 239, right 286, top 1626, bottom -2 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1626) ALTERED SHAPE OF SLICE: V7 final left 239, right 286, top 1627, bottom -3 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1626) ALTERED SHAPE OF SLICE: V7 final left 239, right 286, top 1628, bottom -4 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1626) ALTERED SHAPE OF SLICE: V7 final left 239, right 286, top 1629, bottom -5 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1626) WARNING: may be in infinite loop! ALTERED SHAPE OF SLICE: V7 final left 239, right 286, top 1630, bottom -6 NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1626) shapes did not match! full_cut2slice: (47, 0), corr_vals (47, 1626) working with slice 23 SHAPES (54, 1784) (53, 1784) ALTERED SHAPE OF SLICE: V2 final left 1736, right 1789, top 1784, bottom 0 NEW SHAPE OF SLICE: (53, 1784) and corr_vals.shape: (53, 1784)
working with slice 24 SHAPES (48, 1621) (47, 1621) ALTERED SHAPE OF SLICE: V2 final left 189, right 236, top 1621, bottom 0 NEW SHAPE OF SLICE: (47, 1621) and corr_vals.shape: (47, 1621)
working with slice 25 SHAPES (55, 1790) (54, 1790) ALTERED SHAPE OF SLICE: V2 final left 1784, right 1838, top 1790, bottom 0 NEW SHAPE OF SLICE: (54, 1790) and corr_vals.shape: (54, 1790)
working with slice 26 SHAPES (48, 1617) (47, 1617) ALTERED SHAPE OF SLICE: V2 final left 140, right 187, top 1617, bottom 0 NEW SHAPE OF SLICE: (47, 1617) and corr_vals.shape: (47, 1617)
working with slice 27
SHAPES (55, 1796) (54, 1797)
ALTERED SHAPE OF SLICE: V2
ALTERED SHAPE OF SLICE: V8
final left 1833, right 1887, top 1796, bottom -1
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1797)
ALTERED SHAPE OF SLICE: V8
final left 1833, right 1887, top 1796, bottom -2
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1797)
ALTERED SHAPE OF SLICE: V8
final left 1833, right 1887, top 1796, bottom -3
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1797)
ALTERED SHAPE OF SLICE: V8
final left 1833, right 1887, top 1796, bottom -4
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1797)
ALTERED SHAPE OF SLICE: V8
final left 1833, right 1887, top 1796, bottom -5
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1797)
WARNING: may be in infinite loop!
ALTERED SHAPE OF SLICE: V8
final left 1833, right 1887, top 1796, bottom -6
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1797)
shapes did not match! full_cut2slice: (54, 0), corr_vals (54, 1797)
working with slice 28
SHAPES (48, 1612) (47, 1613)
ALTERED SHAPE OF SLICE: V2
ALTERED SHAPE OF SLICE: V8
final left 90, right 137, top 1612, bottom -1
NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1613)
ALTERED SHAPE OF SLICE: V8
final left 90, right 137, top 1612, bottom -2
NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1613)
ALTERED SHAPE OF SLICE: V8
final left 90, right 137, top 1612, bottom -3
NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1613)
ALTERED SHAPE OF SLICE: V8
final left 90, right 137, top 1612, bottom -4
NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1613)
ALTERED SHAPE OF SLICE: V8
final left 90, right 137, top 1612, bottom -5
NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1613)
WARNING: may be in infinite loop!
ALTERED SHAPE OF SLICE: V8
final left 90, right 137, top 1612, bottom -6
NEW SHAPE OF SLICE: (47, 0) and corr_vals.shape: (47, 1613)
shapes did not match! full_cut2slice: (47, 0), corr_vals (47, 1613)
working with slice 29
SHAPES (55, 1802) (54, 1803)
ALTERED SHAPE OF SLICE: V2
ALTERED SHAPE OF SLICE: V8
final left 1882, right 1936, top 1802, bottom -1
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1803)
ALTERED SHAPE OF SLICE: V8
final left 1882, right 1936, top 1802, bottom -2
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1803)
ALTERED SHAPE OF SLICE: V8
final left 1882, right 1936, top 1802, bottom -3
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1803)
ALTERED SHAPE OF SLICE: V8
final left 1882, right 1936, top 1802, bottom -4
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1803)
ALTERED SHAPE OF SLICE: V8
final left 1882, right 1936, top 1802, bottom -5
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1803)
WARNING: may be in infinite loop!
ALTERED SHAPE OF SLICE: V8
final left 1882, right 1936, top 1802, bottom -6
NEW SHAPE OF SLICE: (54, 0) and corr_vals.shape: (54, 1803)
shapes did not match! full_cut2slice: (54, 0), corr_vals (54, 1803)
Calculating statistics...
Absolute Difference : mean = 0.000e+00 median = 0.000e+00 stdev = 0.000e+00
Maximum AbsoluteDifference = 0.000e+00
Minimum AbsoluteDifference = 0.000e+00
Percentage of pixels where median of absolute differences is greater than:
-> 1xtheshold = 0%
-> 3xtheshold = 0%
-> 5xtheshold = 0%
*** Result of the test: PASSED
*** Final result of pathloss test is PASSED ***
('* Script ifu_uni.py took ', '1.3737628181775412 minutes to finish.')
Did pathloss validation test passed? All slits PASSED path_loss test.
# Quickly see if the test passed
print('These are the final results of the tests: ')
for key, val in results_dict.items():
if not isinstance(val, str):
if val:
val = 'PASSED'
else:
val = 'FAILED'
print('{:<42} {:<8}'.format(key, val))
These are the final results of the tests: ifu_g395h_f290lp_nrs1_uncal.fits All slits PASSED path_loss test. ifu_g395h_f290lp_nrs2_uncal.fits All slits PASSED path_loss test.
Author: Maria A. Pena-Guerrero, Sr. Science Software Engineer, NIRSpec
Updated On: Sep/26/2022